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Abstract. We describe the use of explicit isogenies to translate in- 
stances of the Discrete Logarithm Problem (DLP) from Jacobians of 
hyperelliptic genus 3 curves to Jacobians of non-hyperelliptic genus 3 
curves, where they are vulnerable to faster index calculus attacks. We 
provide explicit formulae for isogenies with kernel isomorphic to (Z/2Z) 3 
(over an algebraic closure of the base field) for any hyperelliptic genus 3 
curve over a field of characteristic not 2 or 3. These isogenies are rational 
for a positive fraction of all hyperelliptic genus 3 curves defined over a 
finite field of characteristic p > 3. Subject to reasonable assumptions, our 
constructions give an explicit and efficient reduction of instances of the 
DLP from hyperelliptic to non-hyperelliptic Jacobians for around 18.57% 
of all hyperelliptic genus 3 curves over a given finite field. We conclude 
with a discussion on extending these ideas to isogenies with more general 
kernels. A condensed version of this work appeared in the proceedings of 
the EUROCRYPT 2008 conference. 

1 Introduction 

After the great success of elliptic curves in public-key cryptography, researchers 
have naturally been drawn to their higher-dimensional generalizations: Jaco- 
bians of higher-genus curves. Curves of genus 1 (elliptic curves), 2, and 3 are 
widely believed to offer the best balance of security and efficiency. This article 
is concerned with the security of curves of genus 3. 

There are two classes of curves of genus 3: hyperelliptic and non-hyperelliptic. 
Each class has a distinct geometry: the canonical morphism of a hyperelliptic 
curve is a double cover of a curve of genus 0, while the canonical morphism of a 
non-hyperelliptic curve of genus 3 is a birational map to a nonsingular plane quar- 
tic curve. A hyperelliptic curve cannot be isomorphic (or birational) to a non- 
hyperelliptic curve. From a cryptological point of view, the Discrete Logarithm 
Problem (DLP) in Jacobians of hyperelliptic curves of genus 3 over ¥ q may be 
solved in 0(q 4 / 3 ) group operations, using the index calculus algorithm of Gaudry, 
Thome, Theriault, and Diem [8]. Jacobians of non-hyperelliptic curves of genus 3 
over ¥ q are amenable to Diem's index calculus algorithm [5], which requires 



only O(q) group operations to solve the DLP (for comparison, Pollard/baby- 
step-giant-step methods require <J(q 3 ^ 2 ) group operations to solve the DLP in 
Jacobians of genus 3 curves over ¥ q ). The security of non-hyperelliptic genus 3 
curves is therefore widely held to be lower than that of their hyperelliptic cousins. 

Our aim is to construct explicit homomorphisms to provide a means of effi- 
ciently translating instances of the DLP from Jacobians of hyperelliptic curves 
of genus 3 to Jacobians of non-hyperelliptic curves, where faster index calculus 
is available. In the context of DLP-based cryptography, we may assume that 
our Jacobians are absolutely simple. In this situation, every nontrivial homo- 
morphism of Jacobians of curves of genus 3 is an isogeny: that is, a surjective 
homomorphism with finite kernel. 

To be specific, suppose we are given a hyperelliptic curve H of genus 3 over 
a finite field ¥ q , together with an instance P = [n]Q of the DLP in Jfj(¥ q ); 
our task is to recover n given P and Q. After applying the standard Pohlig- 
Hcllman reduction [19j . we may assume that P and Q have prime order. We 
want to solve this DLP instance by solving an equivalent DLP instance in a 
non-hyperelliptic Jacobian. Suppose we have an isogeny 4> : Jh — > Jc, where C 
is a non-hyperelliptic curve of genus 3. Further, suppose that 4> is explicit (that 
is, we have equations for C and an efficient map on divisor classes representing (f>) 
and defined over ¥ q , so it maps Jjj(F g ) into Jc(¥ q ). Provided <fr(Q) ^ 0, we can 
recover n by solving the DLP instance 4>(P) — [n](f)(Q) in Jc(F g ) with Diem's 
algorithm. 

The approach outlined above is conceptually straightforward; the difficulty 
lies in computing explicit isogenies of Jacobians of genus 3 curves. Automor- 
phisms, integer multiplications, and Frobenius maps aside, we know of no ex- 
plicit and general formulae for isogenies from Jacobians of hyperelliptic curves 
of genus 3 apart from those presented below. 

In $3] through Sj6l we derive explicit formulae for isogenies whose kernels 
are generated by differences of Weierstrass points, following the construction 
of Donagi and Livne [7]. The key step is making Recillas' trigonal construc- 
tion [20] completely explicit. This gives us a curve X of genus 3 and an ex- 
plicit isogeny Jh —>■ Jx- While X may be hyperelliptic, naive moduli space 
dimension arguments suggest (and experience confirms) that X will be non- 
hyperelliptic with an overwhelming probability, and thus explicitly isomorphic 
to a nonsingular plane quartic curve C . We can therefore compute an explicit 
isogeny <f> : Jh — > Jc', if 4> is defined over ¥ q , then we can use it to reduce DLP 
instances. We note that the trigonal construction (and hence our formulae) does 
not apply in characteristics 2 and 3. 

We show in $5] that, subject to some reasonable assumptions, given a uni- 
formly randomly chosen hyperelliptic curve H of genus 3 over a sufficiently large 
finite field F g of characteristic at least 5, our algorithms succeed in constructing 
an explicit isogeny defined over ¥ q from Jh to a non-hyperelliptic Jacobian with 
probability w 0.1857. In particular, instances of the DLP can be solved in 0(q) 
group operations for around 18.57% of all Jacobians of hyperelliptic curves of 
genus 3 over finite fields of characteristic at least 5. 



We discuss more general isogenies in ^\ Given explicit formulae for these 
isogenics, we expect that most, if not all, instances of the DLP in Jacobians of 
hyperelliptic curves of genus 3 over any finite field could be reduced to instances 
of the DLP in non- hyperelliptic Jacobians. 

Our results have a number of interesting implications for curve-based cryp- 
tography, at least for curves of genus 3. First, the difficulty of the DLP in a 
subgroup G of Jh depends not only on the size of the subgroup G, but upon the 
existence of other rational subgroups of Jh that can be used to form quotients. 
Second, the security of a given hyperelliptic genus 3 curve depends significantly 
upon the factorization of its hyperelliptic polynomial. Neither of these results 
has any parallel in genus 1 or 2. 

The constructions of Sj3] through SjHl an d require some nontrivial alge- 
braic geometry. We have included enough mathematical detail here to enable 
the reader to compute examples, to justify our claim that the construction is 
efficient, and to support our heuristics. 

A Note on the Text 

This article presents an extended version of work that appeared in the proceed- 
ings of the EUROCRYPT 2008 conference [53]. The chief results are the same; 
we have made some (minor) changes to our notation, expanded the derivation 
in SjHl given further details and proofs throughout, and added an appendix with 
algorithms to compute sets of tractable subgroups. 

2 Notation and Conventions for Hyperelliptic Curves 

We will work over ¥ q throughout this article^ where q is a power of a prime p > 3. 
We let Q denote the Galois group Gal(F g /F g ), which is (topologically) generated 
by the q th power Frobenius map. 

Suppose we are given a hyperelliptic curve H of genus 3 over ¥ q . We will use 
both an affine model 

H : y 2 = F(x), 

where F is a squarefree polynomial of degree 7 or 8, and a weighted projective 
plane model 

H : w 2 = F(u,v) 

for H (here u, v, and w have weights 1, 1, and 4, respectively). The coordinates 
of these models are related by x — u/v and y = w/v A . The polynomial F is 
squarefree of total degree 8, with F(u,v) = v 8 F(u/v) and F(x) — F(x,l). 
We emphasize that F need not be monic. By a randomly chosen hyperelliptic 
curve, we mean the hyperelliptic curve defined by w = F(u,v), where F is 
a uniformly randomly chosen squarefree homogenous bivariate polynomial of 
degree 8 over ¥ q . 

1 Some of the theory carries over to more general base fields: in particular, the results 
of Sj5]and §6] are valid over fields of characteristic not 2 or 3. 



The canonical hyperelliptic involution i of H is defined by (x, y) (x, —y) 
in the afhne model, (it : v : w) i— » (u : v : —w) in the projective model, and 
induces the negation map [— f] on Jh- The quotient tt : H — > f/"/ (t) = P 1 
sends (u : v : w) to (u : v) in the projective model, and (x, y) to a; in the affinc 
model (where it maps onto the affine patch of P 1 where t) ^ 0). 

To compute in Jh, wc fix an isomorphism from Jh to the group of degree-0 
divisor classes on H, denoted Pic (H). Recall that divisors are formal sums of 
points in H(¥ q ), and if D = J2peh h p(P) i s a divisor, then J2peH n P * s * ne 
degree of D. We say D is principal if D = div(/) := J2peh or dp(/)(-P) f° r 
some function / on H, where ordp(/) denotes the number of zeroes (or the 
negative of the number of poles) of / at P. Since H is complete, every principal 
divisor has degree 0. The group Pic°(iJ) is defined to be the group of divisors of 
degree modulo principal divisors; the equivalence class of a divisor D is denoted 
by [D]. We let Jh[1] denote the l-torsion subgroup of Jh- that is, the kernel of 
the multiplication-by-l map. If I is prime to q, then J#[Z](F q ) is isomorphic 
to (Z/IZ) 6 . 



3 The Kernel of the Isogeny 

The eight points of H(¥ q ) where w = are called the Weierstrass points of H. 
Each Weierstrass point W corresponds to a linear factor 

L w ■— v(W)u — u(W)v 

of F, which is defined up to scalar multiples. If W\ and Wi are Weierstrass points, 
then 2(Wi)-2(W 2 ) = div{L w J_L W2 ), so 2[{W 1 )-{W 2 )\ = 0; hence {{W 1 )-{W 2 )\ 
represents an element of J H [2] (¥ q ). In particular, [(Wi)-(W 2 )] = [(W2)-(Wi)], 
so the divisor class [(Wi) — (W 2 )] corresponds to the pair {W\, W 2 } of Weierstrass 
points, and hence to the quadratic factor Lyy 1 Lw 2 °f F ( U P to scalar multiples). 

Proposition 1. To every Q -stable partition of the eight Weierstrass points of H 
into four disjoint pairs, we may associate an ¥ q -rational subgroup of Jjj[2](F 9 ) 
isomorphic to (Z/2Z) 3 . 

Proof. Let {{W{, W"}, {W 2 , Wg}, {Wg, W3'}, {W4, W4'}} be a partition of the 
set of Weierstrass points of H into four disjoint pairs. Each pair {W(, W('} cor- 
responds to the 2-torsion divisor class [(W/) — (W/')] in Jh [2](¥ q ). We associate 
the subgroup S := ([(W[) - (Wf )] : 1 < i < 4) to the partition. Observe that 



E[(W/)-aO]= \dW(w/Y[L w ,,) 



= 0; 



i=l i=l 

this is the only relation on the classes [(W() - {W(% so S = (Z/2Z) 3 . The 
action of Q on J^[2](F q ) corresponds to its action on the Weierstrass points, so 
if the partition is ^-stable, then the subgroup S is ^-stable. □ 



Remark 1. By "an F 9 -rational subgroup of Jh [2] (¥ q ) isomorphic to (Z/2Z) 3 " , we 
mean a (/-stable subgroup that is isomorphic to (Z/2Z) 3 over ¥ q . We emphasize 
that the subgroup need not be contained in Jn(¥ q ). 

Remark 2. Requiring the pairs of Weierstrass points in Proposition [1] to be dis- 
joint ensures that the associated subgroup is isotropic with respect to the 2- Weil 
pairing. We will see in SjHlthat this is necessary for the quotient by the subgroup 
to be an isogeny of principally polarized abelian varieties, and hence for the 
quotient to be an isogeny of Jacobians. 

Definition 1. We call the subgroups corresponding to partitions of the Weier- 
strass points of H as in Proposition [7] tractable subgroups. We letS(H) denote 
the set of all ¥ q -rational tractable subgroups of Jh [2](F g ). 

Remark 3. Not every subgroup of Jjj [2](F q ) that is the kernel of an isogeny of 
Jacobians is a tractable subgroup. For example, if W\, . . . , Wg are the Weierstrass 
points of H, then the subgroup 

([(Wi) - (Wi) + (Wj) - (W k )} : k) e {(2, 3, 4), (2, 5, 6), (3, 5, 7)}) 

is a maximal 2- Weil isotropic subgroup of Jjj(¥q) ) and hence is the kernel of 
an isogeny of Jacobians (see However, this subgroup contains no nontrivial 
differences of Weierstrass points, and therefore cannot be a tractable subgroup. 

Computing S(H) is straightforward if we identify each tractable subgroup 
with its corresponding partition of Weierstrass points. Recall that each pair of 
Weierstrass points {W(, W/'} corresponds to a quadratic factor of F (up to scalar 
multiples). Since the pairs are disjoint, the corresponding quadratic factors are 
pairwise coprime, so we may take them to form a factorization of F. We therefore 
have a correspondence of tractable subgroups, partitions of Weierstrass points 
into pairs, and sets of quadratic polynomials (up to scalar multiples): 

S < — ► {{Wi, Wi 1 } : 1 < % < 4} < — ► {Fx,F 2 ,F 3 ,F 4 }, where F = FiF 2 F 3 Fi. 

The action of Q on J#[2](F g ) corresponds to its action on the set of Weierstrass 
points, so the action of Q on a tractable subgroup S corresponds to the action 
of Q on the corresponding set {Fi,F 2 , F 3 , F4} (assuming the have been scaled 
appropriately). In particular, S is F g -rational precisely when {^1,^2,^3,^4} is 
fixed by Q. The factors F, are themselves defined over ¥ q precisely when the 
corresponding points of S are F g -rational. 

We can use this information to compute S(H). The set of pairs of Weierstrass 
points contains a £-orbit ({W£, W^}, . . . , {Wi n ,W" n }) if and only if (possibly 
after exchanging some of the W[ with the W" ) either both (W^ , . . . , W{ n ) 
and {Wi[ , . . . , WfJ are ^-orbits or (W£ , . . . , w[ , W? , . . . , ) is a 0-orbit. 
Every tj-orbit of Weierstrass points corresponds to an F 9 -irreducible factor of F, 
so the size of S(H) depends only on the factorization of F. A table relating 
the size of S(H) to the factorization of F appears in Lemma [T] below; this will 
be useful for our analysis in Sj8l For completeness, we have included a naive 
algorithm for enumerating S(H) in Appendix IA1 



Lemma 1. Let H : w 2 = F(u,v) be a hyperelliptic curve of genus 3 over ¥ q . 
The cardinality of the set S(H) depends only on the degrees of the ¥ q -irreducible 
factors of F, and is described by the following table: 



Degrees of V q - irreducible factors of F 


#S{H) 


(8), (6, 2), (6, 1,1), (4, 2, 1,1) 


1 


(4, 2, 2), (4, 1,1, 1,1), (3, 3, 2), (3, 3, 1,1) 


3 


(4,4) 


5 


(2,2,2,1,1) 


7 


(2,2,1,1,1,1) 


9 


(2,1,1,1,1,1,1) 


15 


(2,2,2,2) 


25 


(1,1,1,1,1,1,1,1) 


105 


Other 






Proof. This is a routine combinatorial exercise after noting that every C/-orbit 
of pairs of Weierstrass points corresponds to either an even-degree factor of F, 
or a pair of factors of F of the same degree. □ 

4 The Trigonal Construction 

We will now briefly outline the theoretical aspects of constructing isogenies with 
tractable kernels. We will make the construction completely explicit in <j5]and $6] 

Definition 2. Suppose S — ([(W/) — (W/')] : 1 < i < 4) is a tractable subgroup. 
We say that a morphism g : P 1 — * P 1 is a trigonal map for S if g has degree 3 
and g(n(W{)) = .9(tt(VK")) for 1 < i < 4. 

Given a trigonal map g for some tractable subgroup S, Recillas' trigonal 
construction [20] specifies a curve X of genus 3 and a map / : X — > P 1 of 
degree 40 The isomorphism class of X depends only on S, and is independent 
of the choice of g (see Recillas [20] . Donagi [6, Th. 2.11], and Remark [5] below) . 
Theorem [IJ due to Donagi and Livne, states that if g is a trigonal map for S, 
then S is the kernel of an isogeny from Jh to Jx ■ 

Theorem 1 (Donagi and Livne [7, §5]). Let S be a tractable subgroup 
in S(H), and let g : P 1 — > P 1 be a trigonal map for S. If X is the curve formed 
from g by Recillas' trigonal construction, then there is an isogeny <f> : Jh — > Jx 
(defined over ¥ q ) with kernel S. 

We will give only a brief description of the geometry of X here, concentrating 
instead on its explicit construction; we refer the reader to Recillas [20] , Vakil [24] , 
Donagi [5] §2], and Birkenhake and Lange [T] §12.7] for proofs and further detail. 

2 Recillas' original trigonal construction is denned where 7r is an etale double cover; the 
trigonal construction we apply here is in fact the flat limit of Recillas' construction 
(see [3 §3] for details). 



The isogeny of Theorem [T] is analogous to the well-known Richelot isogeny in 
genus 2 (see Bost and Mestre [3J, and Donagi and Livne [7J §4] for details), and 
to the explicit isogeny described by Lehavi and Ritzcnthaler in [TJ] for Jacobians 
of non-hyperelliptic genus 3 curves. 

In abstract terms, if U is the subset of the codomain of g above which g o tt is 
unramified, then X is by definition the closure of the curve over U representing 
the pushforward to U of the sheaf of sections of tt : (go 7r) _1 ([/) — * (in 
the etale topology). This means in particular that the F 9 -points of X over an ¥ q - 
point P of U represent partitions of the six F 9 -points of (goTr)~ 1 (P) into two sets 
of three exchanged by the hyperelliptic involution. The fibre product of H and X 
over P 1 with respect to g o tt and / is the union of two isomorphic curves, R 
and R', which are exchanged by the involution on if x P i I induced by the 
hyperelliptic involution. The natural projections induce coverings tth '■ R - *• H 
and ttx '■ R — > X of degrees 2 and 3, respectively, so R is a (3, 2 ^correspondence 
between H and X . 

The maps tth and ttx induce homomorphisms (tth)* '■ Jh — * Jr (the pull- 
back) and (ttx)* '■ Jr — * Jx (the pushforward). In terms of divisor classes, the 
pullback is defined by 

(**)*([£ M^)]) = [£»p £ (Q) 

with appropriate multiplicities where tth ramifies; the pushforward is defined by 

(**)*([ £ m Q (Q)]) - [ m Q (-Kx[Q)) ■ 
Qgr QeR 

Composing (ttx)* with (tth)* \ we obtain an isogeny <fi : Jh — > Jx with kernel S. 
If we replace R with R' in the above, we obtain an isogeny isomorphic to —(f). 
Thus, up to isomorphism, the construction of the isogeny depends only on the 
subgroup S. The curves and Jacobians described above form the commutative 
diagrams shown in Figure [1] 

The hyperelliptic Jacobians form a codimension-1 subspace H. g of the moduli 
space of 3-dimensional principally polarized abelian varieties — which, by the 
theorem of Oort and Ueno [IS] , is also the moduli space M. g of Jacobians of 
genus 3 curves. The Weil hypotheses imply that #Ttg(¥ q )/#M g (F q ) ~ 1/q for 
sufficiently large q (cf. [13j Theorem 1]). In particular, for cryptographically 
relevant sizes of q, the probability that a uniformly randomly chosen curve X of 
genus 3 over ¥ q should be hyperelliptic is negligible. We will suppose that the 
same is true for the curve X constructed in Theorem[T]for a uniformly randomly 
chosen H and S in S(H). This is consistent with our experimental observations, 
so we postulate Hypothesis [T] 



Hypothesis 1 The probability that the curve X constructed by the trigonal con- 
struction for a randomly chosen H/¥ q and S in S(H) is hyperelliptic is negligible 
for sufficiently large q. 



R Jr. 




pi 



Fig. 1. The curves, Jacobians, and morphisms of $4] 



5 Computing Trigonal Maps 

Suppose we are given a tractable subgroup S of Jjj[2](F g ), corresponding to a 
partition {{W 7 /, W"} : 1 < i < 4} of the Weierstrass points of H into pairs. The 
first step in the explicit trigonal construction is to compute a trigonal map g 
for S. We will compute polynomials N = x 3 + n\x + n and D — x 2 + + o?o 
such that the rational map 

_ iV(a;) _ x 3 + mx + n 

defines a trigonal map for S. The derivation is an exercise in classical geometry; 
we include it here to demonstrate its efficiency and to justify Hypothesis^ which 
will be important in determining the expectation of success of our reduction in fjS] 
The reader prepared to admit the existence of efficiently computable trigonal 
maps in the form of |T]) may skip the remainder of this section on first reading. 

By definition, g : P 1 -> P 1 is a degree-3 map with ff(7r(W/)) = g{ir(W")) 
for 1 < i < 4. We will express g as a composition g = p o e, where e : P 1 — > P 3 is 
the rational normal embedding defined by 

e : (u : v) i — > (uq : mi : U2 : U3) = (u 3 : u 2 w : mw 2 : u 3 ), 

and p : P 3 — > P 1 is the projection defined as follows. For each 1 < i < 4, we let Li 
denote the line in P 3 passing through e(7r(W/)) and e(-7r(M / /')). There exists at 
least one line L intersecting all four of the Li (in fact there are two, though 
they may coincide; we will compute them below). We take p to be the projection 
away from L; then p(e(7r(W/))) = p(e(7r(M / /'))) for 1 < i < 4, so g — p o e is a 
trigonal map for S. Given linear equations for L in the coordinates Ui, we can 
use Gaussian elimination to compute elements n\, no, d\, and do of ¥ q such that 



L = V(u + niu 2 + n u 3} U\ + diu 2 + ^0^3) • 



The projection p : P 3 — > P 1 away from L is then defined by 



p : (u : m : u 2 : U3) 1 — ► (ito + niu 2 + "0^3 : u% + d x u 2 + d u 3 ), 
so our trigonal map g — p o e is defined by 

g : (u : v) 1 — > (it 3 + n\uv 2 + Uqv 3 : u 2 v + diuv 2 + dov 3 ). 



Therefore, if we set N(x) := x 3 + n\x + no and D(x) :— x 2 + d\X + d$, then g 
will be defined by the rational map x 1 — > t = N(x)/ D(x). 

To compute equations for L, we will use the classical theory of Grassmannian 
varieties. The elementary Lemmas [2] and [3] will be stated without proof; we refer 
the reader to Griffiths and Harris [9[ §1.5] and Harris [TUl Lecture 6] for details. 

The set of lines in P 3 has the structure of an algebraic variety Gr(l,3), 
called the Grassmannian. There is a convenient model for Gr(l,3) as a quadric 
hypersurface in P 5 : if Vq, . . . , V5 are coordinates on P 5 , then we may take 

Gr(l, 3) := V(v V 3 + v lVA + v 2 v 5 ) C P 5 . 

Lemma 2. There is a bijection between points of Gr(l, 3)(F g ) and lines in P , 
defined as follows. 

1. The point o/Gr(l, 3)(F g ) corresponding to the line through (po : pi : p 2 ■ Pa) 
and (qo : qi : q 2 : 93) in P 3 has coordinates 



Po Pi 



Po Pi 

qo q-2 



PO P3 

qo 93 



P2 P3 
92 93 



P3 Pi 

93 9i 



Pi P2 

9i 92 



The line in 
defined by 



corresponding to a point (70 



75) o/Gr(l,3)(F 9 ) is 



V 



( 0u 

73 "0 

74 "0 

\75"0 



73M1 

J2U1 
7i u i 



74^2 

72""2 

0u 2 
lou 2 



75^3,^ 
71 ""3, 
70 ""3, 

0u 3 J 



(two of the equations will be redundant linear combinations of the others). 

Lemma 3. Let L be the line in P 3 corresponding to a point (70 : • • • : 75) 
of Gr(l, 3)(F g ). The points in Gr(l,3)(F g ) corresponding to lines in P 3 that in- 
tersect nontrivially with L are precisely the points lying in the hyperplane defined 
by Y^i=o7 iVi +3 = (where the subscripts are taken modulo 6). 

Suppose S is represented by a set {Fi — aiu 2 + biuv + av 2 : 1 < i < 4} 
of quadratic factors of F (as in with each factor Fi corresponding to a 
pair {W/, W"} of Weierstrass points. Applying Lemma[21 we see that the line Lj 
through e(ir(W[)) and e(ir(W")) corresponds to the point 



on Gr(l,3). If (70 : • ■ ■ : 7.5) in Gr(l,3)(F g ) corresponds to a candidate for L, 
then by Lemma [3] we have JVf (70, . . . , 75) 



fa\ a\b\ aic% c\ 



M = 



a.3 b 3 



CI2C2 
03 C3 



\a 4 0,4 &4 04 C4 



r 2 
' 4 



= 0, where 
-C161 (b\ - aici)^ 

-c 2 &2 (&! 
-C3&3 (63 



-C4&4 



«2C2 
03C3 

a A c A )) 



(2) 



The kernel of M is two-dimensional, corresponding to a line /l in P 5 . The kernel 
is independent of the ordering of the -F, , and does not change if we replace the Fi 
by scalar multiples; hence, A depends only on the subgroup S. Let {a, /?} be a 
basis for ker M, writing a = (ao, . . . , as) and [3 = ([3q, ■ ■ ■ , ft)- If 5 is F 9 -rational, 
then so is kerM, so we may take the a^ and ft to be in ¥ q (see Cartier (4j §1]). 
We want to find a point Pj, = (ao + A/?o : ■ • • : 05 + Aft) where yl intersects 
with Gr(l, 3). The points (uo U3) on the line L in P 3 corresponding to Pl 

satisfy (M„ + AM^wq, . . . , u 3 ) T = 0, where 



M„ := 



/ 


— a 3 


— 04 


-«5\ 


a 3 





-a 2 


ai 


04 


a 2 







\«5 


-ai 


ao 


0/ 



and M/3 := 



/ 


-03 


-A 


-ft\ 


ft 





-ft 


ft 


ft 


ft 





-ft 


\ft 


-A 


ft 





By part (2) of Lemma [H the rank of AL. + AM/3 is 2. Using the expression 



6 6 6 

det(M a + AM^) =(2(£ ftft+s) ^ + ( E a ^+ 3 ) A + 2 ^ a ' 

i=0 i=0 i=0 



ai+3 



(3) 



(where the subscripts are taken modulo 6), we see that M^ + AM^ has rank 2 pre- 
cisely when det(M«+AM / 3) = 0: we can therefore solve det(Ma+XMp) — to de- 
termine a value for A. Finally, we use Gaussian elimination to compute m, no, d\, 
and do in F 9 (A) such that (1,0, m, no) and (0, 1, di, do) generate the rowspace 
of Mg_ + XMp. We then take L = V(uq + n\Ui + nou^, u\ + d±U2 + dou^), and 
compute p, e, and the trigonal map g = p o e as above. 

Since L is defined over F 9 (A), so is the projection p and the trigonal map g. 
But A satisfies a quadratic equation with coefficients in ¥ q , so F 9 (A) is at most 
a quadratic extension of ¥ q . Computing the discriminant of det(M„ + \Mp), we 
obtain a criterion for existence of trigonal maps over ¥ q for a given tractable 
subgroup. 

Proposition 2. Suppose S is a tractable subgroup, and let {a = {pn),j3 = (ft)} 
be any ¥ q -rational basis of the nullspace of the matrix M defined in @. There 
exists an ¥ q -rational trigonal map for S if and only if 



D 2 

( X! a »ft+s) - ( X! a ^+s) ( ftft+3 



i=0 



i=0 



i=0 



(4) 



is a square in ¥ q , where the subscripts are taken modulo 6. 



Proof. From the derivation above, we see that there exists an F g -rational trigonal 
map for S if and only if we can find a A in ¥ q such that det(M Q + XMp) = 0. 
By Equation (J3]), we can find such a A if and only if the quadratic polynomial 

^ 6 6 ^6 

i=0 i=0 i=0 

has two roots in ¥ q . This occurs precisely when the discriminant of this polyno- 
mial — the expression in ((4]) above — is a square in ¥ q . □ 

Proposition O shows that the rationality of a trigonal map for a tractable 
subgroup S depends only upon whether an element of ¥ q depending only on S is 
a square. It seems reasonable to assume that these field elements are uniformly 
distributed for uniformly random choices of H and S, and indeed this is con- 
sistent with our experimental observations. Since a uniformly randomly chosen 
element of ¥ q is a square with probability ~ 1/2, we propose Hypothesis O 

Hypothesis 2 The probability that there exists an ¥ q -rational trigonal map for 
a subgroup S uniformly randomly chosen from S(H), where H is a randomly 
chosen hyperelliptic curve over¥ q , is 1/2. 

6 Equations for the Isogeny 

Suppose we have a hyperelliptic curve H of genus 3, a tractable subgroup S 
in S(H), and a trigonal map g for S. We will now perform an explicit trigonal 
construction on g to compute a curve X and an isogeny tf> : Jh — ► Jx with 
kernel S. 

We assume that g has been derived as in and in particular that g : P 1 — > P 1 
is defined by a rational map in the form 

Nix) x 3 + n\x + uq 

g '. x i — ► t = = . 

D(x) x 2 + d\x + do 

Observe that g maps the point at infinity to the point at infinity (that is, (1 : 0)). 
For notational convenience, we define 

G(t, x) = x 3 + g 2 (t)x 2 + g x {t)x + g (t) := N(x) - tD{x); 
unless otherwise noted, we will view G(t,x) as an element of F 9 [t][x]. We have 

92 {t) = —t, gi(t) — n\ — d\t, and go(t) = uq — dot. 

We also define /o, /i, and fi to be the elements of ¥ q [t] such that 

fo(t) + fi(t)x + f 2 (t)x 2 = F(x) (mod G(t, x)). 

Let U be the subset of A 1 = P 1 \ {(1 : 0)} above which g o 7r is unramified. 
With the notation above, 

U = Spcc(fcM) \ V((f 2 - 4/ 2 /o)(4. 92 3 . 9o - g\g\ - 18g 23 i.9o + k)\ + 27g 2 )) . 



We will derive equations for an affine model X\u of / _1 (L/) — that is, the open 
subset of X over U. We will not prove here that the normalization of X\u is 
isomorphic to the curve X specified by Recillas, but we will exhibit a bijection on 
geometric points. If X is not hyperelliptic, then taking the canonical map of X\jj 
into P 2 will give us a nonsingular plane quartic curve C isomorphic to X. 

By definition, every point P in X\jj(¥ q ) corresponds to a pair of unordered 
triples of points in H(¥ q ), exchanged by the hyperelliptic involution, with each 
triple supported on the fibre of g o 7r over f(P). To be more explicit, suppose Q 
is a generic point of U. Since g o n is unramified above Q, we may choose three 
preimages P\ , P 2 , and P 3 of Q such that 



Viewing unordered triples of points as effective divisors of degree 3 (that is, as 
formal sums of three points), we have 



Note that Pi and t(Pi) never appear in the same divisor for any 1 < i < 3. 
There is a one-to-one correspondence between effective divisors of degree 3 on H 
satisfying this condition, and ideals (a(x),y — b(x)) where a is a monic cubic 
polynomial and b is a quadratic polynomial satisfying b 2 = F (mod a) (this is 
the well-known Mumford representation 17, §IIIa]). For example, Pi + P2 + P3 
corresponds to the ideal (a(x),y — b(x)) where a(x) = Yii( x ~ x (Pi)) and b 
satisfies y{Pi) = b(x(Pi)) for 1 < i < 3 (with appropriate multiplicities); we may 
compute b using the Lagrange interpolation formula. A divisor is defined over ¥ q 
if and only if a and b are defined over ¥ q . The ideal (a(x),y — b(x)) corresponds 
to P1+P2 + P3 if and only if (a(x) , y + b(x)) corresponds to t(Pi) + t(P 2 ) + i(P 3 ); 
so each point of X over U corresponds to a pair {(a(x),y ± b(x))} of ideals. 
We will construct a curve parametrizing these pairs of ideals, and take this as a 
model for X\jj. 

Suppose {(a(x),y ± b(x))} is a pair of ideals corresponding to one of the 
preimages of Q on X\u- The product of the two ideals is equal to the principal 
ideal (a(x)); but products of ideals correspond to sums of divisors, so (a(x)) 
must cut out the divisor P1+P2 + P3 + l(Pi) + ^2) + 1(^3) on H. This divisor 
is just (g o ir)*(Q), which we know is cut out by (G(t(Q),x)); so we conclude 
that a(x) = G(t(Q),x) for every pair of ideals {(a(x),y ± b(x))} corresponding 
to a point in f^ 1 (Q). In particular, the generic point of X\u corresponds to a 
pair of ideals of the form {(G(t, x), y ± (bo + b\X + 62X 2 ))}, where b , bi, and b 2 
are algebraic functions of t such that 



(g o iry l (Q) = {Pi, P 2 , P 3 , t(Pi), l(P 2 ),l(P 3 )}. 



f-\Q) 



Qi «-> (P a + P 2 + P 3 , t(P x ) + l(P 2 ) + l(P 3 ) 
Q2 «-> (Pi + t(P 2 ) + t(P 3 ), i(Pi) + P2 + P3 
Q3 " U(Pi) +P2 + t(P 3 ), A + ^2) + P3 
Qi ~ {t(Pi) + t(P 2 ) + P 3 , ^1 + ^2 + t(P 3 ) 




(5) 



(6 + ^ia; + &2a; 2 ) 2 = P(a;) (mod G(t,x)). 



(6) 



Viewing 60, 61, and 62 as coordinates on A 3 (over ¥ q ), we expand both sides 
of ([6|) modulo G(t, x) and equate coefficients to obtain a variety X in [/ x A 3 



parametrizing ideals: 

X = V(co(t,b ,bi,b2),ci(t,bo,bi,b2),C2(t,bo,bi,b 2 )) , 

where 

c (t, b , 61, b 2 ) = g 2 (t)go(t)b 2 2 - 2g (t)b 2 b 1 + b 2 - f (t), 

c 1 (t,bo,b 1 ,b 2 ) = {g 2 (t)gi(t) ~9o(t))bl - 2g 1 (t)b 2 b 1 + 2hb - h(t), and 

c 2 (i, 60, 61, b 2 ) = {g 2 (t) 2 - gi{t))bl - 2g 2 (t)b 2 b 1 + 2b 2 b + b 2 - f 2 (t). 

The ideals in each pair {(G(t,x),y ± (b 2 x 2 + b\x + bo))} are exchanged by 
the involution /,* : X — > X defined by 

(.* : (t, bo, h,b 2 ) 1 — > (t, -b , -61, -62); 

the curve is therefore the quotient of X by (*,*). To make this quotient 
explicit, let m : U x A 3 — > [/ x A 6 be the map defined by 

m : (t, bo,bx, b 2 ) 1 — > (t, 6 o, b i,bo 2 ,b 11 ,b 12 ,b 22 ) = (t, b 2 , b b 1} b b 2 , b\, hb 2 , bj); 

observe that 

m(U x A 3 ) = v(f m ~ bm ^ 5ol , 6o r b °t 12 u hl \2 b °t 22 u )dUxA e . 

VO02O11 - 0oi0l2, O02O12 - O01&22, H 2 -bnb 22 J 

We have = rn(X), so 

(c (i, &00, ■ • ■ ,&22),ci(t,&oo, ■ • • ,b 22 ),c 2 (t, boo, ■ ■ -,b 22 ), 
&01 - &00&11, ^01&02 - ^00^12, &02 - &00&22, ) C U X A 6 , 

602^11 — &01&12, ^02^12 — boib 22 , b\ 2 — b\\b 22 

where c , Ci, and c 2 are the polynomials defined by 



co(t, b o, boi, 602, 611, h 2 ,b 22 ) 
ci(t, b o, boi,b 02 , bn, 612, 622) 
c 2 (i, &00, b i, b 02 , hi, 612, 622) 



= g 2 gob 22 - 2g Q b 12 + boo - fo, 

= (5251 - go)b 22 - 251612 + 26 i - fi, and 

= (g 2 - gi)b 22 - 2g 2 b 12 + 2b 02 + 611 - f 2 . 



Observe that X\jj is defined over the field of definition of g. 

It remains to derive a correspondence R between H and X\jj inducing the 
isogeny </>. We know that R is a component of the fibre product HxpiX (with 
respect to g o it and /). We may realise the open affine subset H\jj Xjj X\u as 
the subvariety V(G(t,x)) of H \u x X\u; decomposing the ideal (G(t,x)) will 
therefore give us a model for R. 

Lemma 4. Let s be the polynomial in V q [t] defined by 

s := /o 3 - flfm - 2f 2 f 29l + f 2 f 2 g 2 + foff 9l + 3/0/1/2.90 - /o/i/20i<fc (7) 
- 2/ / 2 2 5off2 + /o/ 2 2 5i " fi9o + A 2 /2ffo52 - fifhm + Ihl, 

and let a be its leading coefficient. Then s has a square root in ¥ q (y/a)[t]. 



Proof. The polynomial s is a square in F g (-y/a)[t] if and only if each of its roots 
in ¥ q occur with multiplicity 2. In the notation of we have 

s(t(Q)) = F(x(P 1 ))F(x(P 2 ))F(x(P 3 )), 

so s(t(Q)) = if and only if F(x(Pi)) = for some 1 < i < 3 — that is, if and 
only if at least one of the Pi is a Weierstrass point of H . But the trigonal map g 
was constructed precisely so that the Weierstrass points of H appear in pairs 
in the fibres of g: hence exactly two of the Pi must be Weierstrass points, and 
so F(x(Px))F(x(P 2 ))F(x(P 3 )) = and s(t(Q)) = with multiplicity 2. □ 

Proposition 3. Let s be the polynomial of Lemma^ and let So, S±, 5 2 , and 84 

be the polynomials in ¥ q 2 [t] defined by 

84, := -27gl + 18g gig 2 - 4g 0ff | - Agf + g\g\, 

S 2 := 12/o.gi - 4/ o5 | - 18/i.g + 2fi9i92 + ^f 2 g g 2 - 4f 2 gf, 

5i := 8-y/s, and 

So ■= -4/o/ 2 + fl 

On the curve X\u, we have 

(S 4 (t)b 2 22 + S 2 (t)b 22 + S (t)) 2 - 5 x {tfb 22 = 0. (8) 

Proof. Consider again the fibre of / : X — > P 1 over the generic point Q — (t) of U 
(as in {5}). If {Pi+P 2 +P3, l(Pi)+l(P 2 )+i(P3)} is a pair of divisors corresponding 
to one of the points in the fibre, then by the Lagrange interpolation formula the 
value of b 22 at the corresponding point of X is 

b 22 =(^y{P l )/{(x{P l )-x{PM^)-<P^))\ (9) 

where the sum is taken over the cyclic permutations (i,j,k) of (1,2,3). After 
interpolating for each pair of divisors in the fibre, an elementary but involved 
symbolic calculation shows that b 22 satisfies 

(^ 2 -2(^r i )6 22 + i(2(^^)-(^r i ) 2 )) 2 -64(nA)6 22 =0, (10) 

i i i i 

where 

r t := {f 2 {t)x{P t f + fi(t)x(Pi) + /o(t)) A = F(x(Pi))Ai 

with 

A := (x(Pj) - x{P k )f 

for each cyclic permutation (i,j,k) of (1,2,3), and where A := AiA 2 A 3 . 

Now A, J2iFi, ^2iTi, and Yl i Ti are symmetric functions with respect to 
permutations of the points in the fibre g~ 1 (Q) — g -1 ((£)). They are therefore 
polynomials in the homogeneous elementary symmetric functions 

ei = 2jx(Pi), e 2 = } j x(P i )x(P j ), and e 3 = rjx(P), 

i i<j i 



which are polynomials in t. Indeed, the ej are given by the coefficients of G(t 1 x): 

ei = ~92{t), e 2 = gi(t), and e 3 = -go(*)- 

Expressing Z\, £\ Y,i r h and IL-f» in terms 01 /o, /i> /2, So, 9i, and g 2 , 
and substituting the resulting expressions into (TlT)|) , we obtain jSJ). □ 

Equation ([5]) gives us a (singular) affine plane model for X. We can also 
use © to compute a square root for &22 on we have 

h 2 h 6,(^+62(^22+60^) 

b 2 2 = p , where p := — . 

Returning to ©, we observe that 622 is a unit on X|(7, since its zeroes and poles 
occur only at points Q where g o 7r is ramified over f(Q), and these points were 
excluded from [/. Since p is the square root of &22, it must also be a unit on X\jj. 

Given a point (t, 600, ■ ■ ■ , ^22) of X\u, the corresponding pair of divisors of 
degree 3 on H is cut out by the pair of ideals 

{(G(t,x),y±( b -f + b fx+ b fx>))}. 

This is precisely the decomposition of (G(t 7 x)) that we need to compute the 
correspondence from H\u to X\u' we have V(G(t, x)) = R\J R', where 

R = v(G(t,x),y- ^(b 02 + b 12 x + b 2 2X 2 )^j (11) 

and 

R' = V (c(t, x),y+- p (b 02 + b 12 x + 6 22 .x 2 )^ . 

On the level of divisor classes, the isogeny <j> : J H — > J x is made explicit by the 
map 

4> = (nx)* o (tth)*, 

where wh ■ R H and ttx : R —* X\u are the natural projections defined 
by (x, y,t,b 00 ,..., 622) h-> (x, y) and (x, y, t, b 00 , b 22 ) i-> (*, &00, • • • , h.2), re- 
spectively. In terms of ideals cutting out effective divisors, <f> is realized by the 
map 

Id •— > (ir> + (G(t,x),y- - p (b Q 2 + b 12 x + b 22 x 2 )^ n¥ g [s,t, boo, ■ ■ ■ , b 22 ]- 

Taking R' in place of R in the above gives an isogeny equal to —(f). It remains 
to determine the field of definition of <f>. 

Proposition 4. If S is a subgroup in S(H) with an ¥ q -rational trigonal map g 
defined over ¥ q , and s(t) is the polynomial defined in Lemma^ then the explicit 
trigonal construction on g described above yields an isogeny defined over ¥ q if 
and only if the leading coefficient of s(t) is a square in ¥ q . 



Proof. We noted earlier that X\u is defined over the field of definition of g. The 
correspondence R, and hence the induced isogeny cf>, are both defined over the 
field of definition of p, which is the field of definition of 6461 , 82 /Si , and 60/81. 
But 64, 64, and <5o are all defined over ¥ q (cf. Proposition [3]), while <5i is defined 
over F 9 (i/a) where a is the leading coefficient of s by Lemma [4] □ 

Remark 4- If </> is not defined over F 9 , then the Jacobian Jx is in fact a quadratic 
twist of the quotient Jh/S (see $9]). In fact, when (f> is not F g -rational, Frobenius 
exchanges p and —p, hence R and R 1 , and therefore <f> and —<fi. This is a concrete 
realization of the Galois cohomology referred to in the proof of Proposition [5] 
below: the obstruction to the existence of an isomorphism from Jh/S to Jx 
over ¥ q is in fact the interaction of Q with [±1] on Jx- 

If we assume that the leading coefficients of the polynomials s(t) are uni- 
formly distributed for randomly chosen H , S, and g, then the probability that s 
is a square in ¥ q [t] is 1/2. Indeed, it is easily seen that s(t) is a square for H if and 
only if it is not a square for the quadratic twist of H . Suppose H : w 2 = F(u, v) is 
a hyperelliptic curve. Let c be a non-square in ¥ q , and let H' : w 2 — cF(u, v) be 
the quadratic twist of H. Suppose S in S(H) is a tractable subgroup, represented 
by a set {Fi, F 2 , F 3 , F4} of quadratic factors of F. The set {cFi, F 2 , F 3 , F4} is a 
factorization of cF, so it represents a tractable subgroup 5" in S(H'). We noted 
in fJUthat scalar multiples of quadratic polynomials do not affect the construc- 
tion of trigonal maps; so if S has a trigonal map g defined over F 9 , then g is 
also a trigonal map for S' . Let s be the polynomial computed from g and S in 
Lemma |4l and let s' be the corresponding polynomial computed for g and S". 
Looking at the form of 0, we see that s'(t) = c 3 s(t). Therefore, the leading 
coefficient of s' is a square if and only if the leading coefficient of s is not a 
square. In particular, if 5* has a trigonal map defined over F 9 , then so does S", 
and we can construct an isogeny of Jacobians with kernel S if and only if we 
cannot construct an isogeny of Jacobians with kernel S'. 

This suggests that the probability that we can compute an isogeny defined 
over F g given a randomly chosen H and S in S(H ) with a trigonal map defined 
over ¥ q is 1/2 — since we have a 50% chance of being on the "right" quadratic 
twist of H. This hypothesis is consistent with our experimental observations. 

Hypothesis 3 For a randomly chosen hyperelliptic curve H and a uniformly 
randomly chosen subgroup S in S(H) with a trigonal map g defined over ¥ q , the 
probability that we can compute an ¥ q -rational isogeny <f> with kernel S is 1/2. 

7 Computing Isogenies 

Now we will put the ideas above into practice. Suppose we are given a hyperellip- 
tic curve H of genus 3 over ¥ q , and a DLP instance in J# (F g ) to solve. Our goal is 
to compute a nonsingular plane quartic curve C and an explcit isogeny Jh — > Jc 
defined over ¥ q , so that we can solve our DLP instance in Jc(¥ q ). 



We begin by computing the set S(H) of F g -rational tractable subgroups of 
the 2-torsion subgroup Jh [2](F g ) (see Appendix 1X1 below) . For each S in S(H), 
we apply Proposition^ to determine whether there exists an Fq-rational trigonal 
map g for S. If so, we use the formulae of f|5]to compute <?; if not, we move on to 
the next S. Having computed g, we apply Proposition [4] to determine whether 
we can compute an isogeny over ¥ g . If so, we use the formulae of ^to compute 
equations for X and the isogeny (p : Jh — > Jx\ if n ot, we move on to the next S. 

The formulae of $5] give an affine model of X in A 1 x A 6 . In order to apply 
Diem's algorithm to the DLP in Jx , we need a nonsingular plane quartic model 
of X: that is, a nonsingular curve CcP 2 isomorphic to X, cut out by a quartic 
form. Such a model exists if and only if X is not hyperelliptic. To find C, we 
compute a basis B = {ipi, ip2, ips} °f the Riemann-Roch space of a canonical 
divisor of X. This is a routine geometrical calculation; Hess [TT] describes an 
efficient approach. In practice, the algorithms implemented in Magma |2ll5j 
compute B very quickly. The three functions in B define a map ip : X — > P 2 , 
mapping P to {ip\{P) : ip2 (P) '■ 4>3(P))- Up to automorphisms of P 2 , the map ip 
is independent of the choice of basis B, and depends only on X. If the image of ip 
is a conic (that is, if the ipi satisfy a quadratic relation), then X is hyperelliptic; 
in this situation we move on to the next S, since we will gain no advantage from 
index calculus on X . Otherwise, the image of ip is a nonsingular plane quartic C, 
and ip restricts to an isomorphism ip : X — > C. 

If the procedure outlined above succeeds for some S in S(H), then we have 
computed an explicit F g -rational isogeny ip* o <p : J H — > J c . We can then map 
our DLP from Jff(F g ) into Jc(F 9 ), and solve it using Diem's algorithm. 

We emphasize that the entire procedure is very fast: the curve X and the 
isogeny can be constructed using just a few low-degree polynomial operations 
and some low-dimensional linear algebra (and hence the procedure is polynomial- 
time in logg, the size of the base field). For a rough idea of the computational 
effort involved, given a random H over a 160-bit prime field with a tractable 
subgroup 5* in S(H), a naive implementation of our algorithms in Magma com- 
putes the trigonal map g, the curve X, the nonsingular plane quartic C, and the 
isogeny <p : Jh — > Jc m a f ew seconds on a 1.2GHz laptop. Since the difficulty of 
the construction depends only upon the difficulty of arithmetic in ¥ q (and not 
upon the size of the DLP subgroup of J#(F g )), we may conclude that instances 
of the DLP in 160-bit Jacobians chosen for cryptography may also be reduced 
to instances of the DLP in non-hyperelliptic Jacobians in very little time. 

Example 1. We will give an example over a small field. Let H be the hyperelliptic 
curve over F37 defined by 



Using the ideas in fJH or the algorithms in Appendix [A] we find that Jh has 
one F37-rational tractable subgroup: 



H :y 2 



x 7 + 28x 6 + 15x 5 + 20x 4 + 33x 3 + 12a; 2 



+ 29x + 2. 




where & is an element of F 37 3 satisfying £f+29£?+9£i + 13 = 0, and = £f 0100 . 
Applying the methods of $5j we compute a trigonal map g : x i— ► N(x)/D(x) 
for S 1 , taking 

2V(a;) = .t 3 + 16a; + 22 and D(x) = x 2 + 32x + 18; 

clearly g is defined over F37. The formulae of J)] give us a curve X C A 1 x A 6 of 
genus 3, defined by 



X = V 



I (18t 2 + 15t)&22 + (36t+30)bi2+&oo + 19t 5 + 10t 4 + 12t 3 +7t 2 +t+30, N 
(32t 2 +2t+15)6 2 2 + (27t+5)6i2+26 i+5i 5 +26t 4 + 15t 3 +23t 2 + 19t+17, 
(t 2 +32t+21)f)22+2tfci2+2&02+;)ii+36t 5 +29t 4 +7t 3 + 13t 2 +21t+18, 

Y f)Q0bl boo &12-f<01 602, 600622 -bo2> fe 02&ll-&Olf)12,fco2f)12-fcoi &22, & 2 2 - & l 1^22 / 



The map on divisors inducing an isogeny from Jh to Jx with kernel S is induced 
by the correspondence R defined as in pip with 

G(t, x) = x 3 - tx 2 - {32t - 16)x - 18t + 22, 

S = 27t w + 20t 9 + m s + (St 7 + mt 6 + 8t 5 + 9i 4 + 2t 3 + 31t 2 + 15t + 16, 

81 = 35t 3 + 8t 2 + m + 3, 

8 2 = 20t 7 + I8t 6 + 29t 5 + Ut 4 + 6t 3 + 20t 2 + I2t + 16, and 
84 = 27t 4 + 36t 3 + 13t 2 + 21t. 

Computing the canonical morphism of X, we find that X is non-hyperelliptic, 
and isomorphic to the nonsingular plane quartic curve 



C = V 



u A + 26u 3 v + 2u 3 w + I7u 2 v 2 + 9u 2 vw + 20u 2 w 2 + 34uv 3 + 2Auv 2 w 
+ huvw 2 + 36uw 3 + 19w 4 + 13v 3 w + v 2 w 2 + 23vw 3 + 5w 4 



Composing the isomorphism with the isogeny Jh — > Jx, we obtain an explicit 
isogeny <f> : Jh — > Jc- We can verify that Jh and Jc are isogenous by checking 
that the zeta functions of H and C are identical: indeed, direct calculation with 
Magma shows that 

„.„ _ 37 3 T 6 + 4 ■ 37 2 T 5 - 6 ■ 37T 4 - 240T 3 - 6T 2 + 4T + 1 
Z(H;T) = Z{C-T) = ______ . 

Let D = [(10 : 28 : 1) - (14 : 6 : 1)] and D' = [(19 : 28 : 1) - (36 : 13 : 1)] be 
divisor classes on H; we have D 1 = [22359]D. Applying tj>, we find that 

(p(D) = [(7 : 18 : 1) + (34 : 34 : 1) - (18 : 22 : 1) - (15 : 33 : 1)] and 
4>{D') = [(7 : 23 : 1) + (6 : 13 : 1) - (13 : 15 : 1) - (7 : 18 : 1)] ; 

direct calculation verifies that <f>(D') = [22359] 4>(D), as expected. 



8 Expectation of Existence of Computable Isogenics 



Our aim in this section is to estimate the proportion of genus 3 hyperelliptic 
Jacobians over F g for which the methods of this article produce an F 9 -rational 



isogeny — and thus for which the DLP may be solved using Dicm's algorithm 
— as q tends to infinity. We will assume that if we are given a selection of ¥ q - 
rational tractable subgroups of a given Jacobian, then the probabilities that each 
will yield a rational isogeny are mutually independent. This hypothesis appears 
to be consistent with our experimental observations. 

Hypothesis 4 For a randomly chosen hyperelliptic curve H , the probabilities 
that we can compute an ¥ q -rational isogeny with kernel S for each S in S(H) 
are mutually independent. 

Theorem 2. Assume Hypotheses @ @ and [7} As q tends to infinity, the 
expectation that the algorithms in this article will give a reduction of the DLP 
in a subgroup of Jni^q) for a randomly chosen hyperelliptic curve H of genus 3 
over ¥ q to a subgroup of Jc(¥ q ) for some nonsingular plane quartic curve C is 

E (C 1 " C 1 " V4) S(T) )/ II M») ! ' ^ rH )) « 0.1857, (12) 

TeT n£T 

where T denotes the set of integer partitions of 8 and vt(p) denotes the mul- 
tiplicity of an integer n in a partition T , and s(T) = f^S{H), where H is any 
hyperelliptic curve over¥ q such that the multiset of degrees of the ¥ q -irreducible 
factors of its hyperelliptic polynomial coincides with T. 

Proof. Suppose H is a randomly chosen hyperelliptic curve of genus 3 over ¥ q . 
Hypotheses [U [21 and [3] together imply that for each S in S(H), the probability 
that we can compute an isogeny with kernel S defined over ¥ q is 1/2-1/2-1 = 1/4. 
Hypothesis 2] implies that we have an equal chance of constructing an isogeny 
from each S in S(H), so the probability that we can compute an isogeny over ¥ q 
from Jh is 1— (1 — 1/ 4)# S ( H \ The expectation that we can compute an isogeny 
over ¥ q given a curve over ¥ q is therefore 

Ej;(l-(3/4)** g >) 

h i := v^T~i ' V-^) 

where H is the curve defined by w 2 — F(u,v), and F ranges over the set of 
all homogeneous squarefree polynomials of degree 8 over ¥ q . Lemma [1] implies 
that #S(H) depends only on the degrees of the F 9 -irreducible factors of F, so 
the map T i— > s(T) is well-defined. For each T in T, let N q (T) denote the number 
of homogeneous squarefree polynomials over ¥ q whose multiset of degrees of F g - 
irreducible factors coincides with T. We can now rewrite (|13|) as 

Sr e r(l-(3/4) 8(r) )^(T) 
Z TeT N q (T) 

There are N q (n) = i Yldln A*(^)9 n monic irreducible polynomials of degree n 

over F 9 (here n is the Mobius function). Clearly N q {T) = (q-l) H neT (^("j), 
so 

N g (T) = ( [] (Mn)l ■ n^)) q 9 + 0(g 8 ), 



and X^TeT N q (T) = q 9 + 0(q 8 ). Therefore, as q tends to infinity, we have 

1™,^= E ( (l - (3/4) s « r >) / II (Mn)\-n^)). 

'' v TeT neT 

The result follows upon explicitly computing this sum, using the values for s(T) 
listed in Lemma [TJ □ 

Theorem[2]gives the expectation of our ability to construct an explicit isogeny 
for a randomly selected hyperelliptic curve. However, looking at the table in 
Lemma [TJ we see that we can be sure that a particular curve has no isogenies 
with tractable kernels defined over ¥ q if we use only curves whose hyperelliptic 
polynomials have an irreducible factor of degree 5 or 7 (or a single irreducible 
factor of degree 3) . It may be difficult to efficiently construct a curve in this form 
if we are using a CM construction, for example, to ensure that the Jacobian has a 
large prime-order subgroup. In any case, it is interesting to note that the security 
of genus 3 hyperelliptic Jacobians depends significantly upon the factorization 
of their hyperelliptic polynomials. This observation has no analogue for elliptic 
curves or Jacobians of curves of genus 2. Of course, if E : y 2 = F(x) is an 
elliptic curve and F is completely reducible, then #E(¥ q ) is divisible by 4, and 
in particular #E(¥ q ) cannot be prime; but this does not reduce the security 
of E(¥ q ) to the extent that a completely reducible hyperelliptic polynomial does 
for a curve of genus 3. 

Remark 5. We noted in Jj2] that the F g -isomorphism class of the curve X in the 
trigonal construction is independent of the choice of trigonal map. If there is no 
trigonal map defined over ¥ q for a given subgroup S in S(H), then the methods 
of SjSJconstruct a pair of Galois-conjugate trigonal maps g\ and 172 (corresponding 
to the roots of (jSJ)) instead. Applying the trigonal construction to g\ and g%, we 
obtain curves X\ and X2 over ¥ q 2. If the isomorphism between X\ and X2 
were made explicit, then we could descend it to compute a curve X over ¥ q in 
the F g -isomorphism class of X\ and X2 , and hence a nonsingular plane quartic C 
over ¥ q and an isogeny Jh ~+ Jc- We note that the isogeny may not be defined 
over ¥ q , but this approach could still allow us to replace the 1/4 in (|13|) and (|12p 
with 1/2, raising the expectation of success in Theorem[2]to 31.13%. 

Example 2. Let p = 1008945029102471339. Note that pis a 60-bit prime; if 
H is a hyperelliptic curve of genus 3 over F p such that Jh{¥ p ) has a large 
prime-order subgroup and if Gaudry-Thome-Theriault-Diem index calculus is 
the fastest algorithm for solving DLP instances in J#(F p ), then Jh has roughly 
the same security level as an elliptic curve over a 160-bit field. 

We generated one million random hyperelliptic curves of genus 3 over F p us- 
ing Magma. For each curve H, we computed the set S(H) of tractable subgroups; 
then, for each S in S(H) we determined whether there was an F p -rational trigo- 
nal map for S, and if so whether there was an F p -rational isogeny with kernel S. 
Of these curves, 502005 (that is, 50.02%) had at least one rational tractable sub- 
group. Between them, the 10 6 curves had 1002244 rational tractable subgroups, 



of which 501629 had a rational trigonal map (that is, 50.05%, which is close to 
the 50% predicted by Hypothesis [2]). Of these subgroups, 250560 led to a rational 
isogeny (that is, 49.95%, which is close to the 50% predicted by Hypothesis [3]) . 
We found that 185814 of the curves had at least one F p -rational isogeny, none 
of which had a hyperelliptic codomain (this is compatible with Hypothesis [T]) . 
In particular, we could move a discrete logarithm problem for 18.58% of these 
curves (recall that Theorem [5] predicts a success rate of about 18.57%). 

9 Other Isogenies 

So far, we have concentrated on using isogenies with kernels generated by differ- 
ences of Weierstrass points to move instances of the DLP from hyperelliptic to 
non-hyperelliptic Jacobians. More generally, we could use isogenies with other 
kernels. There are two important issues to consider here: the first is a theoret- 
ical restriction on the types of subgroups that can be kernels of isogenies of 
Jacobians, and the second is a practical restriction on the isogenies that we can 
currently compute. 

Let H be a hyperelliptic curve of genus 3. We want to characterize the sub- 
groups S of Jh that are kernels of isogenies of Jacobians, combining standard 
results from the theory of abelian varieties with some special results on curves 
of genus 3. For our purposes, it is enough to know that the l-Weil pairing is a 
nondegenerate, bilinear pairing on the /-torsion of an abelian variety, which can 
be efficiently evaluated in the case where the abelian variety is the Jacobian of 
a hyperelliptic curve; for further detail, we refer the reader to [T2l Ex. A. 7. 8]. 

Definition 3. Let A be an abelian variety over ¥ q , and let I be a positive integer 
coprime with q. We say a subgroup S of A[l] is maximal /-isotropic if 

1. the l-Weil pairing on A[l] restricts trivially to S, and 

2. S is not properly contained in any other subgroup of A[l] satisfying (1). 

If I is a prime not dividing q, then every maximal /-isotropic subgroup 
of Jjj{¥ q )[l] is isomorphic to (Z//Z) 3 . The situation is more complicated when / 
is not prime: for example, Jy[2] is a maximal 4-isotropic subgroup of Jh [4], but 
it is isomorphic to (Z/2Z) 6 and not (Z/4Z) 3 . 

Proposition 5. Let H be a hyperelliptic curve of genus 3 over¥ q such that Jh 
is absolutely simple. Let S be a finite, nontrivial, ¥ q -rational subgroup of Jh(¥ q ). 
There exists a curve X of genus 3 over ¥ q , and an isogeny 4> : Jh —* Jx with 
kernel S, if and only if S is a maximal l-isotropic subgroup of Jh[1] for some 
positive integer I. The isogeny <J> is defined over ¥ q 2. 

Proof. The quotient Jh — > Jh / S always exists as an isogeny of abelian varieties, 
and is defined over ¥ q (see Serre .21, §111.3.12]). For the quotient to be an isogeny 
of Jacobians, there must be an integer I such that S is a maximal /-isotropic 
subgroup (see Proposition 16.8 of Milne [16]): this ensures that the canonical 
polarization on Jh induces a principal polarization on the quotient Jh/S. The 



theorem of Oort and Ueno [TH] therefore guarantees that there will be an iso- 
morphism of principally polarized abelian varieties over ¥ q from Jh/S to the 
Jacobian Jx of some irreducible curve X (irreducibility of X follows from the 
fact that Jh, and hence Jh/S, is absolutely simple). Composing this isomor- 
phism with the quotient map gives an isogeny of Jacobians from Jh to Jx with 
kernel S. Standard arguments from Galois cohomology (see Serre [331 PII.l], 
for example) show that the isomorphism is defined over either ¥ q or ¥ q 2 , and it 
follows that the isogeny Jh — > Jx must be defined over F g or ¥ q 2 . □ 

Remark 6. Proposition [5] does not hold in higher genus: for every g > 4, there 
are (/-dimensional abelian varieties that are not isomorphic to Jacobians. Indeed, 
this is the generic situation: for g > 2 the moduli space of (/-dimensional abelian 
varieties is g(g + l)/2-dimensional, with the Jacobians occupying a subspace of 
dimension (3g — 3) — which is strictly less than g(g + 1) /2 for g > 4. We should 
not therefore expect an arbitrary quotient of a Jacobian to be isomorphic to a 
Jacobian in genus <? > 4. Proposition [5] does hold in genus 1 and 2, and in these 
cases the isogenies are always defined over ¥ q . 

We can expect the curve X of Proposition [5] to be non-hyperelliptic. To com- 
pute an F ? -rational isogeny from Jh to a non-hyperelliptic Jacobian, therefore, 
the minimum requirement is an F g -rational Z-isotropic subgroup of Jh(F 9 ) iso- 
morphic to (Z/ZZ) 3 for some prime I. We emphasize that this subgroup need 
not be contained in J#(F 9 ). Indeed, there may be isogenies from Jh to non- 
hyperelliptic Jacobians over ¥ q even when Jh {¥ q ) has prime order (which would 
be the desirable situation in cryptological applications). 

The major obstruction to using more general isogenies to move DLP instances 
is the lack of general constructions for explicit isogenies in genus 3. Apart from in- 
teger multiplications, automorphisms, Frobenius isogenies, and the construction 
for isogenies with tractable kernels exhibited above, we know of no constructions 
for explicit isogenies of general Jacobians of genus 3 hyperelliptic curves. In par- 
ticular, while we know that the curve X of Proposition [5] exists, we generally 
have no means of computing a defining equation for it, let alone equations for a 
correspondence between H and X that would allow us to move DLP instances 
from Jh to Jx- This situation stands in marked contrast to the case of isoge- 
nies of elliptic curves, which have been made completely explicit by Velu |25j . 
Deriving general formulae for explicit isogenies in genus 3 (and 2) remains a 
significant problem in computational number theory. 

Acknowledgements 

The greater part of this work was completed in the Department of Mathematics 
at Royal Holloway, University of London, where the author was supported by 
EPSRC grant EP/C014839/1. The author gratefully acknowledges Roger Oyono 
and Christophe Ritzenthaler for discussions which inspired this research, and 
Steven Galbraith and the anonymous referees for their helpful suggestions. 



References 



1. Birkenhake, C, Lange, H.: Complex abelian varieties (2e), Grundlehren der math- 
ematischen Wissenschaften 302. Springer (2004) 

2. Bosnia, W., Cannon, J., Playoust, C: The Magma computational algebra system. 
I. The user language. J. Symbolic Comp. 24(3-4), 235-265 (2006) 

3. Bost, J.-B., Mestre, J.-F.: Moyenne arithmetico-geometrique et periodes des 
courbes de genre 1 et 2. Gaz. Math. Soc. France 38, 36-64 (1988) 

4. Cartier, P.: Isogenics and duality of abelian varieties, Annals of Mathematics 71 
no. 2, 315-351 (1960) 

5. Diem, C: An index calculus algorithm for plane curves of small degree. In: Hess, 
F., Pauli, S., Pohst, M. (eds.) ANTS- VII. LNCS, vol. 4076, pp. 543-557. Springer 
(2006) 

6. Donagi, R.: The fibres of the Prym map. In: Curves, Jacobians, and abelian 
varieties (Amherst, MA, 1990). Contemp. Math. 136, 55-125 (1992) 

7. Donagi, R., Livne, R.: The arithmetic-geometric mean and isogenies for curves of 
higher genus. Ann. Scuola Norm. Sup. Pisa CI. Sci. (4) 28 no. 2, 323-339 (1999) 

8. Gaudry, P., Thome, E., Theriault, N., Diem, C: A double large prime variation 
for small genus hyperelliptic index calculus. Math. Comp. 76, 475-492 (2007) 

9. Griffiths, P., Harris, J.: Principles of Algebraic Geometry. Wiley and Sons (1978) 

10. Harris, J.: Algebraic Geometry: A First Course. Springer (1992) 

11. Hess, F.: Computing Riemann-Roch spaces in algebraic function fields and related 
topics. J. Symbolic Computation 33 v.4 425-445 (2002) 

12. Hindry, M., Silverman, J.: Diophantine geometry: an introduction. Graduate 
Texts in Mathematics 201. Springer (2000) 

13. Lang, S., Weil, A., Number of points of varieties in finite fields. Am. J. Math., 
LXXVI No. 4 819-827 (1954) 

14. Lehavi, D., Ritzenthaler, C: An explicit formula for the arithmetic geometric 
mean in genus 3. Experimental Math. 16 421-440 (2007) 

15. The Magma computational algebra system, http://magma.maths.usyd.edu.au/ 

16. Milne, J. S.: Abelian varieties. In: Arithmetic geometry (Storrs, Conn., 1984), pp. 
103-150. Springer (1986) 

17. Mumford, D.: Tata Lectures on Theta II. Birkhauser (1984) 

18. Oort, F., Ueno, K.: Principally polarized abelian varieties of dimension two or 
three are Jacobian varieties. J. Fac. Sci. Univ. Tokyo Sect. IA Math. 20, 377-381 
(1973) 

19. Pohlig, G., Hellman, M.: An improved algorithm for computing logarithms 
over GF(p) and its cryptographic significance. IEEE Trans. Info. Theory 24, 106- 
110 (1978) 

20. Recillas, S.: Jacobians of curves with </ 4 's are the Prym's of trigonal curves. Bol. 
Soc. Mat. Mexicana (2) 19, no. 1 9-13 (1974) 

21. Serre, J.-P.: Algebraic Curves and Class Fields. Graduate Texts in Mathematics 
117. Springer (1988) 

22. Serre, J.-P.: Galois Cohomology. Springer (2002) 

23. Smith, B.: Isogenies and the discrete logarithm problem in Jacobians of genus 3 
hyperelliptic curves. In: Smart, N. (ed.) EUROCRYPT 2008. LNCS vol. 4965, 
pp. 163-180. Springer (2008) 

24. Vakil, R.: Twelve points on the projective line, branched covers, and rational 
elliptic fibrations. Math. Ann. 320 no. 1, 33-54 (2001). 

25. Velu, J.: Isogenies entre courbes elliptiques. C. R. Acad. Sci. Paris, Series A 273, 
305-347 (1971) 



A Appendix: Computing S(H) 



Given a hyperelliptic curve H of genus 3 over ¥ g , we want to compute the 
set S(H) of F 9 -rational tractable subgroups of Jh- Algorithm 0] splits the hyper- 
elliptic polynomial of H into Galois orbits of factors, before calling the recursive 
subroutine Algorithm [5] to enumerate S{H). This algorithm is included only for 
completeness, and is not particularly efficient (we suggest some optimisations in 
Remark [7] b elow . ) 

Algorithm 4 Given a hyperelliptic curve H of genus 3 over ¥ q , enumerates the 
setS(H) ofFq-rational tractable subgroups of Jh [2] (F g ) . Each subgroup inS(H) 
is represented by a set of four coprime quadratic factors of F. 

Input The hyperelliptic polynomial F(u,v) of H . 
Output The set S(H). 

Step 1 Let T be the set of irreducible factors of F over its splitting field, 

scaled so that F = YIle^^' an< ^ se ^ ® := {}• 
Step 2 Choose a polynomial L from T . Set O :— (L), set T := T \ {L}, 
and set L\ := L. 

Step 3 Set L := o~(L), where a denotes the q th power Frobenius map. 

If L =/= L±, then append L to O, set J- := T \ {£}, and go to Step 3. 

IfL = L 1; then set O := O U {O}; if T ^ 0, then go to Step 2. 
Step 4 Return the result of Algorithm^ applied to O. 

Algorithm 5 Given a set of Q -orbits of coprime linear polynomials over ¥ q , 
returns the Q -invariant sets of coprime quadratic products of the polynomials. 

Input A set O of disjoint sequences of distinct linear polynomials. Each se- 
quence O = (Oi, . . . , O m ) in O must satisfy 0\ = cr(O m ) and Oi+i = cr(Oi) 
for 1 < i < m, where a denotes the q -power Frobenius map. 

Output The set S of Q -stable sets of coprime quadratic polynomials such that 

Uses IIqgs Q = Iloeo Ulgo l - 
Step 1 If O is empty, then return S :— {0}. 

Step 2 Choose a sequence O from O, and set m := #0. 

If m is even, then let T be the result of Algorithm^ applied to O \ {O}, 

and set S := {{0. t ■ (m/2)+i : 1 < i < m/2} U T : T e T}. 

If m is odd, then set S :— {}. 
Step 3 For each P inO\ {O} such that #P = #0 = m, 

Step 3i Set U := {{Oi +l ■ P 1+ ^ i+j ) mod m) : < i < to} : < j < m} . 

Step 3ii Let V be the result of Algorithm^ applied to O \ {O, P}. 

Step 3iii Set S := S U {U U V : U S U, V £ V}. 
Step 4 Return S. 

Remark 7. As we noted above, Algorithms 4 and 5 are not particularly efficient: 
for conceptual simplicity we worked over the splitting field of the hyperelliptic 
polynomial, and this can be extremely slow in practice. A number of simple 
optimizations will significantly improve the performance of this algorithm: the 



key is to avoid field extensions where possible, and to minimize their degree in 
any case. Before factoring F over its splitting field we should factor it over ¥ q , 
and then work on a case-by-case basis depending on the degrees of the ¥ q - 
irreducible factors. For example, if F has an odd number of odd-degree factors, 
then S(H) is empty by Lemma [TJ and we can simply return the empty set. If F 
is F g -irreducible, then it is not necessary to factor F over its splitting field (which 
is F ? s): there is one tractable subgroup, and it corresponds to the four quadratic 
factors of F that we obtain by factoring F over V q i . Making similar modifications 
for the cases where F has factors of degree 6, we can avoid working over any 
extensions of degree greater than 4. If desired, we can further avoid some field 
extensions in the case where F has only low-degree factors. These modifications 
resulted in a factor-of-50 speedup for our experiments with 60-bit prime fields; 
the unmodified Algorithms [4] and [5] should not be used in practice. 



